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Big-O 
What's the big picture? 


Big-O is a technique that lets you 
quickly understand an algorithm's 
efficiency. 

“This function has a big-O of N? which 
means that if you pass in N items to it, it 
will take roughly N? steps to process its 

data.” 
Big-O is not exact - instead, it gives 
rough estimates in terms of well-known 
functions like log,(N), N, N*log,(N), or N2. 


We use the Big-O approach to quickly 
compare different algorithms and pick the 


HOW Fast IS [nat 
Algorithm? 


Sometimes we want to ask “How fast is that 
—!=-rithm?” 
i Or: “Which algorithm is 
faster, A or B?” 


Question: How can you measure the 
“speed” of an algorithm? 


HOW Fast IS [nat 
Algorithm? 


Right! We could measure the time 
it takes for an algorithm to run. 


But that has flaws! 
What are they? 


Carey: “My algorithm finished in 31 seconds.” 
Cedric: “Mine finished in 30 seconds, it’s better!” 


Carey: “Not so fast! How fast is your PC? Mine is LGHZ.” 
Cedric: “Err... Mine is 3GHZ.” 
Carey: “Aha - so my algorithm is really almost 3x 


Rii 


i — faster!” | i 
Cedric: “Sigh. Carey's right again.” 


HOW Fast is Inat 
OK, so simon th Wn? time of an 


algorithm isn’t really all that useful. 


What if instead we measure an algorithm based on: 


how many computer instructions it takes 
to solve a problem of a given size 


Carey: “My algorithm took 370 million 
Instructions to sort 1,000 numbers.” 


Cedric: “Dude - you SUCK! Mine only took only 5 
million instructions, it’s better!” 


Carey: “Not so fast grasshopper! Mine might be slower 
on 1,000 numbers, but what if we sort 1 million 


i numbers?” i 
Cedric: “Hmm. I don’t know - | haven't tried.” 


' HOW Fast IS [nat 
Algorithm? 


So just rating an algorithm based on how many steps 


it takes on a particular set of data doesn't tell us 


much. 
An algorithm might look efficient when applied to a 


small amount of data (e.g., 1,000 numbers) 


But really “blow chunks” when applied to a lot of data 


(e.g. 1 billion numbers) 


We'd like to understand how our 
algorithm performs under all 
circumstances! 


HOW Fast IS [nat 
Algorithm? 


Hmmm. What else could we do? 


Right! What if we specify 
the number of instructions used by an algorithm 
as a function of the size of the input data. 


“I'm trying to sort: N numbers.” 
“Algorithm A takes 5-N? instructions to do that.” 
“Algorithm B tak 37,000-N ion instructions to do 
that.” 


Now we can predict which algorithm will be faster 
for any value of N! 


HOW Fast IS mn 


“l'm Algo to thm? numbers.” 


“Algorithm A takes 5:N2 instructions to do that.” 
“Algorithm B takes 37,000-N instructions to do that.” 


Ok, what if we're sorting 1,000 numbers: 
“Algorithm A takes 5M instructions. "if 


“Algorithm B takes 37M instructions.” 

Ok, what if we’re sorting 10,000 numbers: 
“Algorithm A takes 500M instructions.” 
“Algorithm B takes 370M instructions. T 


Ok, what if we're sorting 1 million numbers: 


“Algorithm A takes 5 trillion instructions.” _ 


( 
“Algorithm B takes 37 billion instructions.” A 


HOW Fast IS mn 


“l'm Algo to Kith? numbers.’ 


“Algorithm A takes 5-N2 instructions to do t ia e? 
“Algorithm B takes 37,000-N instructions to do = 


Cool! When we measure this way, we get two | 
1. We can compare two algorithms for a given sized inp 


2. We can predict the performance of those algorithms 
when they are applied to less or more data. 


This is the idea behind the “Big-O” concept used in 
Computer Science. 


No, not Oprah. Let’s learn the details! 


Big-O: The Concept 


The Big-O approach measures an algorithm 
by the gross number of steps that it requires 
to process an input of size N H, 
in the WORST CASE scenario. WORST cacy 


SC. 
ival Handpe RIO 


We could be specific and say: 
“Algorithm X requires 5N2+3N+20 steps to process N items.” 


But with Big-O, we 
ignore the coefficients and lower-order terms of the 


expression... 
So we'd Say: 


“The Big-O of Algorithm X is N2.” 


While less specific, this still gives us an overall 
impression of an algorithm’s worst-case efficiency. 
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" É > 
Big-O: The This is pronounced: 
Big-O Idea: Use simple funct 


log(n), n3, etc. to convey hoy “Oh of n squared” 


algorithm must perform to a monec worst 
case. 


“That sorting algorithm is O(n2), so to sort n=1000 
items it requires roughly 1 million operations.” 


“That sorting algorithm is O(n-log,n), so to sort n=1000 
items requires roughly 10,000 operations.” 


This allows us to easily compare two different 
algorithms: 


“Algorithm A is O(n2), which is much slower than 
algorithm B which is O(n:log,n).” 
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Big-O 
So how do we compute the Big-O of a function? 


First, we need to determine the number of 
operations an algorithm performs. Let's call this 
f(n). 


By operations, we mean any of the following: 


1. Accessing an item (e.g. an item in an array) 
2. Evaluating a mathematical expression 
3. Traversing a single link in a linked list, etc... 


Let's see how to evaluate the number of operations 
for a simple example... 


" BIg-O - HOW to Compute 


n} 
| ( Compute f(n), the # of 
int arr[n][n]; critical operations, that this 


for ( inti=fQ l din: i- +) algorithm performs? 
for(int = dij <f; j++ 
aril] = 0 


) 
f(n) =H nh m nt n?+ n% n? 
f(n) = 3n? +3n + 1 
1. Our algorithm initializes the value of i once. 
. Our algorithm performs n comparisons between i and n. 
3. Our algorithm increments the variable /, n times. 
4. Our algorithm initializes the value of j, n different 
Gm@ur algorithm performs n? comparisons between | 
@ncdwr algorithm increments the variable j, n? times. 
7. Our algorithm sets arr[i][j]’s value n? times. 


ow that we have f(n), we can compute our algorithm’s Big-C 


i Big-O - The Complete 
Approach 


Here are the steps to compute the Big-O of an 
algorithm: 


1. Determine how many steps f(n) an algorithm 
requires to solve a problem, in terms of the 
number of items n. 


2. Keep the most significant term of that function and 


throw away the rest. For example: 
a. f(n) = 3n2+3n+1 becomes f(n) = 3n? 
b. f(n) = 2n log(n) + 3n becomes f(n) = 2n log(n) 


3. Now remove any constant multiplier from the 
function: 
c. f(n) = 3n? becomes f(n) = n? 
d. f(n) = 2n log(n) becomes f(n) = n log(n) 

4. This gives you your “big oh”: 


a ffin) = 2n2+2n+7 ic therefore Oln2) 
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Big-O Simplification 


Actually, if you think about it, there's no need to 
compute the exact f(n) of an algorithm... 


Why? Because we end up throwing away all of the 
lower-order terms and coefficients anyway! 


All you need to do is focus on the most frequently 
occurring operation(s) to save time! 


Int arr[n][n]; 


- i 
arr[i][j] = O 
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Big-O 

So if I say: “This algorithm is O(n2).” 

| really mean: “To process n items, this algorithm 

requires 

roughly n? operations.” 


By using only the 
most significant term 


(e.g. n2 from S 
2n?+3n+1) 
S 
We can quickly obtain 
a rough approximation 
an 109 nsn 


of how many steps 
our algorithm will take 


to process n items. 
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Big-O Complexity 


logn IN nlog,n m PB 2” 

3 10 30 100 1000 1000 
6 100 600 10,000 1,000,000 103° 
9 1,000 9,000 1,000,000 1,000,000,000 | 10391 
13 10,000 130,000 100,000,000 | 10? wow! 
16 100,000 1,600,000 10! 101 WOW! 
19 1,000,000 19,000,000 102? 1018 wow! 


What if you wanted to use an O(n?) algorithm to sort a 
million numbers? Your algorithm would require roughly 
1,000,000,000,000,000,000 steps! 


But an O(n log,(n)) algorithm would use only 19,000,000 


Big-O Complexity 
1,000,000,000,000,000,000 vs 19,000,000! 


GREAT PROGRAMMERS know that the choice of 
algorithm makes all the difference in the world. 


NOT-SO-GREAT programmers think that you can 
tweak a poor algorithm to make it better! 


Say you improve an O(n?) algorithm from f(n) = 5n? steps to f(n) = 
1.5n? steps. For n=1,000,000, that reduces the number of steps 
from 5,000,000,000,000,000,000 to 1,500,000,000,000,000,000. 


(Big deal... so it'll take 1.5 years to run instead of 5 years) 


However, if you can find an algorithm that’s O(n:log n) steps, say 
f(n) = 10-n log n, you can solve the problem in 190,000,000 steps. 


(Which will take just a few seconds or less on a modern PC!) 


Don't be a Big-O NUT! 


When you're writing a program that operates ona 
large number of items, evaluating Big-O is key. It can 
mean the difference between a usable program and 


an unusable one. . 
But what if you have a small number of items, e.g. 


n<50? 
In this case, all of your 
algorithms require only 
a small number of 2 
steps. = 
In such situations S 


(when you know n is 

small), forget which 

Big-O is better and (n)=n log "Fon)=n 
choose the easiest-to- 

program algorithm. 

It’ll save you lots of 


hanna rhanc 
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K 


for ( 
SUI, 


for (int | = 0; | < q; I+ 
for ( int j = 0; j 
sum+ 


while (k > s 

{ z =i; 
sur 
k = t=0 ; i < n*n*n; i++) 

} | um += j; 
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Pil LE DIY UV UIIAGIETIYE, Fart 
la 


Int searchArray(int arr[], int n, int forValue) 


for (int! = O; i < n; i+ 
{ 
if (arr[i] == forValue) 
return |: 


} 


return -1; // not found 


} 


void additemToEndOfAr@y(int arr[], int &n, int addMe) 


arr[n] = addMe: We run the same, fixed 
n=n + 1: number of steps no matter what 
) n’s value is. So it's O(1). 


* _BIQ-U... 


Sometimes you'll rdn into an algorithm 
that isn’t so clear-cut. For example, 
what’s the Big-O of mystery? 


It’s clear that the outer loop runs n 
times, but what about the inner 


loop? eae pen 
When q = 0, the inner loop runs 0 iterations. 50 WNat S the Dig- 
When q = 1, the inner loop runs 1 iteration. å 
When q = 2, the inner loop runs 2 iterations. f(n) = nn 
When q = n-1, the inner loop runs n-1 iterations. 22 
O(n?) 


So the cout statement will run a total of: 


0 times + 1 time + 2 times + 3 times + … + n-1 times 
And if you recall a clever trick, this is equal to: ie 


Big-O: Such Ugly Math! [] 
But we're not Math geeks... We're CS geeks! So here's 
a way to address these situations without formulas! 


E But this loop runs a 
Locate all loops tha Variable mom bero f iterations 
and determine the iterati E 


This outer loop always runs 


exactly n iterations (a fixed 
Turn these loops in #). 


using th © Them 
id T i 
© There's nothing to change. ¿> / 


ex 


Let's just round that up to 
n, and then make our 
loop a fixed loop! 


O(n?)In that case, this loop will 
run up to n? iterations - 
Let’s update it! 


Find the Bia-O® salt 


The number of 


The number of CS 


people, p, and the students, C, 
number of foods, f, and EE students, e, 
are completely are 
independent. also independent. _ 
void buffet(string people[], int p, void tinder(string csmajors[], int c, 
string foods[], int f) string eemajors[], int e) 
{ 
int i, j; for (int i=0; i < c ;ji++) 
for (int j=0; j < c ;j++) 
for (i=0; i < p ;i++) cout << csmajors[i] << “ 
for §=0; j < f ;j++) dates “ 
cout << people[i] << “ ate’ << csmajors[j] << ẹ A 
| << foods[j] << er, N ea ee 7 
cout A at 
7 OX 
hat? O(c? + e) and not just O(c?) ? FE 
Int ) i ic’) m's Big- 


fe)... «| Correct! Even though c? seems 
bigger than e, we must include 
both independent variables in our 
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Big-O for Multi-input Algorithms 


Why must we include both variables in the Big-O 
(even if one is higher-order than the other)? 


Because either variable could dominate the other! 


VO 


| 


void tinder(string csmajors[ 


string eemajors[], int e) 


for (int i=0; i < c ;i+ C terate ai 
for (int j=0; j < c ;j+ L 
cout << csmajors[i] << “ 
dates “ 
<< csmajors|/ re eranen 
for (int m=0; m <c; m 
cout << csmajors[m] <<" is still a nere 
ions. 
e iterat! r 
for (int k=0; k < e ;k++ x 


cout << eemajors[k] <<" s 


home“ 


], int c, 


VL 
| OS 


re 10 Dill 


SEG 


Ay K f 
dey yms fpr 
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lable! 


Find the Big-O Challenge, Part 3 


void bar( int n, int q ) void burp( int n ) 
{ 


for (int i=0 ; i < n*n; i++) Ro n iterations AIF eaee 


{ JKE: Keler Mal ger-Uahahaha!”; 
for (ig run of 
2 iterations: n; i++) 
} +o) feria 
} 


for (inti=O;1<n; i++ 
cout << “Muahahaha 


for (int i for (int k = 0; k < q; k+ 
cout << “Muahahaha!”; 


else 
cout << “Burp!”; 


The STL FULLER Does that remind you of 


Remember the STL - stacks, ( a search algorithm that we 


Well, these classes use algé 


( 
And these dia AL] 


learned on the first day of class? 


d 


void inDict(set<stri le, if we want to 

w) Any guesses why it tak 

{ if ( d.find(w) == 0 n steps to delete the first 
cout << w << item from a vector 


icti [rs ae i 
BIS HODSR ER containing n items? 


nt to add a value 

to the end of a vector holding 

n items, it takes just one step, 
so It’s O( 1 )! 


void otherFunc(vect 
{ vec.push_ba 


if we want to delete the 1st value from a vector 
containing 


n tame Tt tabane a AKA nnninn Kn etnne maLinn it F\(n\ 


word in a set that 


The STL and Bia-O 


It's ee woe WE Se cearna eh descr 
(e.g. push_back, erase) n items for a single value vector) 
: requires log.(n) steps... 
… because without know q Jain) step asses, 
we can’t compute the Big-O of at uses the STL classes! 
void | t(set<string> & d, string 
For example... w) 
{ if ( d.find(w) == d.end() ) 
lf we write a loop of our cout << w << “isn’tin 


own that runs D times... dictionary!”; 
And we repeat this search | 


And each iteration of our 
loop searches for an item 


r 


operation D different times... ) 


in 
a set holding n items... for (int i=0; i < D ; i++) 
Thee thehaigsGraf Bia-O hole inDict( dict, doc[i] ); 


loop Wout bb 77? log,(n) ). |) 
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Comp Ok, ourvector  ) 77 
contains q 
values... 
C A 
void pri s( vector<int> & 
{ 


cout << a; 
v.erase( v.begin() 
item 


o K use STL 
And as such, our 


lo 
Is each time our fee) 
>» 


2. Erase the first item in 
the vector: the Big-O of 


5 


3. Add the item to the end 

of the vector: the Big-O of 

adding an item to the end 
of a vector is O(1). 


v.push_back(a); 
} 


// add it to end 


q * (1+ q+ 1) 
So our total Big-O is: 
O(q?) 


Total steps performed during our toop: 


Hint: 
The STL vector 
Insert at the top/middle: O(n 
Insert an the end: O(1 


Delete an item from top/middle:  Of(r 
Delete an item from the end: 
Finding an item: 


DIL and big ON Cheat 
Sheet 


When describing the Big-O of each operation (e.g. 
insert) on a container (e.g., a vector) below, we 
assume that the container holds n items when the 
operation is performed. 


Name: list F 
Purpose: Linked list 

Usage: list<int> x; x.push_back(5); 
Inserting an item (top, middle*, or bottom): O(1) 
Deleting an item (top, middle*, or bottom): O(1) 


Accessing an item (top or bottom): O(1) 
Accessing an item (middle): O(n) 
Finding an item: O(n) 


T 


ema, HER ae RAK R OA you do your 


en N peme 
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So if our loop runs a total of q 2 
CE iterations… Igor” ok, "I assume BER 


set always has q items in 
and each iteration we insert afte it. 
at a cost of log,(q) into our »Ur Sé 
set... /alue That means that each 
time we insert an item 
e mē , i 
Then our total costisq * s it into our set it takes 


~ log»(q) FRE + logz(q) steps. 


lon i ka ould we use as 
= cost of inserting an 


item into the set? 


A a per into a set with n items 


takes log,(n) steps. 


And in fact, our set will have a 
And that’s the correct different number of items during 
answer! each iteration of the loop! 


a Well, assuming our À 
vector has p items and 
we delete one item per 
iteration, our loop runs 


for p steps. 
v aang d 


/ So there are p total loop > 
iterations, and during each 
iteration, we perform p 
steps to delete the first 
item: 


~ O(p?) d 


{ 


{ 


item 
} 
} 


ume p items in vector v 
clearFromFront(vector<int> &v) 


while ( v.size() > 0) 


v.erase( v.begin() ); // erase 1% 


Hints: 


The STL vector 
Insert at the top/middle: 


the eng 


Delete an item from the end: 
Access an item: 
Finding an item: 


O(n) 
O(1 


Insert an 
elete an item from top/middle: O( 


O(1) 
O(1) 
O(n) 


During each step, we delete the © 
vector’s first item. 


Let’s assume the vector always 
has p items, no matter what. 


Cost of deleting first item: O(p) / 


” So we perform q? total loop ^ 
iterations, and during each 
iteration, we perform 
log-(q7) steps to insert an 
item: 


C O(g2"log,(q2)) d 


a 


q) 
{ 


„As before, to co e the eee 


an STL operation, we assume the 
set always holds its max # of 
items. 
In this case, the set will 
eventually hold q? items - that’s 


I 

Delete an tern tron the end: O(1) 
Access an item: O(1) 

Finding an item: O(n) 


// Assume s starts out empty 
void additems(set<int> &s, int 


Ok, our loop 
runs through q 
total iterations 


for (int i=0; i < ENE 


ON 


2 


d 


) 


And the cost of adding a 


single item 
items is. 


.. O(log,(q?)) 


to a set with q? 


O(log,n) 

Finding an item: 
O(log.n) 

Deleting an item: 


Inserting a new item: 


O(log,n) 


7 rena 
Well, assuming our a So there are z total loop > 
vector has z items and iterations, and during each 
we delete one item per iteration, we perform 1 step 
iteration, our loop runs to delete the last item: 
for z steps. 
p | | O(Z) d 
sume z items in vector 
clearFromBack(vector<int> &v) 
while ( v.size() > 0) 
{ 
v.pop_back(); // erase last item 
} 
} 
Hints: 
The STL vector During each iteration, we delete ` 
Insert at the top/middle: O(n) the vector’s last item. 
Insert an the end: O(1) 


Delete an item from top/middle: O(n Let's assume the vector always 
Delete an item from the end: O(1) has z items, no matter what. 


Access an item: O(1 


Finding an item: O(n) Cost of deleting last item: O(1) / 


Find the Big-O Challenge, Part 5 


— Q —> 
oa I have a vector of sets of ints: 
teed vector< set<int> > vi: 
{7 Bae} 


Ace You may assume vector v has N total sets in it. 
| Be Koy may assume that each set has an avg of Q items. 


{ } 
(l1,0,1,...} Questions: 


What is the Big-O of determining if the 
first set, v[0], contains the value 7? 


What is the Big-O of determining if any set 
in v has a value of 7? 


What is the Big-O of determining the 

number of even values in all of v? 
What is the Big-O of finding the first s¢ 1Sd 
7 and then counting the numbe 
set? ia 


As the big-g 


© 
AR as a f 


void reve Susrayll, int n) 


$ 


int tmp, i; 


for (i = 0; i < n/2; ++i) 
{ 
tmp = arrayl[il]; 
array[i] = array[n-i-1]; 
array[n-i-1] = tmp; 
} 
} 


Space Complexity: 
or 


Int *tmparr = new int[n]; 
for (int i = 0; | < n; ++i) 
tmparr[n-i-1] = array[i]; 


for (int | = 0; | < n; ++i) 
array[i] = tmparr[i]; 


delete [] tmparr; 


Space Complexity: 


This function uses just 
memory slots 
no matter how big n is. 


Space Complexity: 


| // prints from n down-to ( 
// 


/ void printNums(int n) 


Vi . 
// prints from n down-to 0 j /| { if (n < 0) return; 
I r V cout << n << “\n”; 
void printNums(int n) i { r printNums(n-1); 
: int i { r printwums(n-L); 

int i; l 


for (i=n; i >= O; i--) printNums(n-1); 


cout << ji << “\n”; 


PMIMUIVUTTIS( I=L; 


The recursive version creates a 
for 


printNums(1000); 


} 


Inefficient Sorting Algos. 


Flow knowing Wa 
the 
complexities of 
different 
sorting 
algorithms is 


Inefficient Sorting Algorithms 
What’s the big picture? 


Sorting is the process of ordering a bunch 
of values, e.g., from smallest to largest. 


There are a handful of “inefficient” 
algorithms to do sorting, like selection 
sort, insertion sort and bubble sort. 


These algorithms generally require O(N?) 
steps to order N values. They’re suuuper 


Slow. 
Why slow? They compare every item to 


every other item, swapping out-of-order 
items. 


Pre-Sorting Intermission 


Brought to you by last year's students who claimed this lecture was too boring) 


Sorting! 


Sorting is the process of ordering a bunch of items 
based on one or more rules, subject to one or more 


constraints... 

Items - what are we sorting, 
and how many are there? 
+ Strings, numbers, student 

records, 

C++ objects (e.g., Circles, 

Robots) 
RUIRGusAanasy AMiBHS OF BlidhsemM ? 
' Ascending” vs. Descendhag 

order 
+ Based on Circle radius? Student 

GPA? 

Based on multiple criteria, e.g.: 


Conssyqaathame, then first name 
+ Are the items in RAM or on disk? 


e Is the data in an array or a linked 
list? 


Carey’s 2 Rules 
of Sorting 


Rule #1: 
Don’t choose a sorting algorithm until you 
understand the requirements of your 
problem. 


Rule #2: 


Always choose the simplest sorting algorithm 
possible that meets your requirements. 
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The Selection sort 


- Look at all N books, select the 
shortest book 


- Swap this with the first book 
- Look at the remaining N-1 
books, and select the shortest 
- Swap this book with the 
second book 


- Look at the remaining N-2 
books, and select the 


third book and so on... 


if we Ian N voks, how many 
Steps Goes it fake to sort them? 


Let’ Sjassume a step is any time 
"wd eitherfswap a book or 
pofnt Our finger at a book. 


The Selection Sort- Speed 


N steps 
1 step 
N-1 steps 


1 step 
N-2 steps 
1 step 


So this comes to: 
N swap steps 
PLUS 


N+N-1+N-2+...4+2+1 
steps to find the smallest item 


~ 


i 


So Selection Sort is 
O(N2) 


Or, for N books, you need 
roughly N? steps to sort 
them. 


(It’s considered pretty 


) 


Selection Sort - Better or Worse? 


Are there any kinds of input data where Selection 
Sort is either more or less efficient? 


For example, what if all of 
the books are mostly in 
order before our sort starts? 


void selectSort(shelf of N 
books) 
{ 

for i= 1toN 


find the smallest book 
between slots i and N 

swap this smallest book No! selection sort 
with book i; takes just as many 

} steps either way! 


The Se Selection Sort Questions 


Can Selection Sort be applied easily to 
And here sort items within a linked list? 


code 


Is Selection Sort “stable” or “unstable”? 


ÆT" 


void selectionSort(int A[], int n) 
{ 
for (int i = 0; i < n; i++) _ For 6 
{ elen 
int minIndex = i; 
for (int j = itl; j < n; j++) 
{ 
if (A[j] < A[minIndex]) 
minIndex = j; 


} 
swap(A[i], A[minIndex]); =! 


What's al Unstable sort says: b 


Imagine that N old people | “Grandpa, you need to chill. 
Everyone's been ordered based on 
And the drugstore "ans aii the number of days they haven’t 
ee ee ee E pooped. That’s totally fair! Geta 
Al Carey says: ~ life.” 4 


AYS: 
“I’m not happy, either! Why y i without taking into 


did Andrea get moved in line jjnce | 

before me? I know we've efore re fair.” 
both been constipated for5 ki, | 

C days, but I was ahead in before 


linge” fter the 
Peon ne 3 ' Ebeneezer says: \ 
carey <5 0 8 day Steve - 8 days 


As PNIS proble s (Mass da% work) yous Mun 
VOL bro endi i SIGE ility pay 
il sat aoe p rea rey a a ti we 
ve - 8 day and V, 3 
vilicvous farget the CONC PAN jure reqagmbe there ty Baan 
O 5 days David - 2 days Davidme'!days p 


Dunt the initial 
of similar-valued items. 


NYE © 


52 
The Se Selection Sort Questions 
Can Selection Sort be applied easily to 
And here sort items within a linked list? 
code Is Selection Sort “stable” or “unstable”? 
to Q When might you use Selection Sort? P 
void selectionSort(int A[], int n) | >» 
{ Here’s a hint - 
for (int i = 0; i < n; i++) considof this array: 
et | 10/10] 1 
int minIndex = 1; 
for (int j = itl; j < n; j++) When Selection Sort 
finds the 1, it swaps it 
if (A[j] < A[minIndex]) with the first 10. 
minI 2j. 
} Then our array ends 
swap(A[i], A[minIndex]); up like this: 
} 1/10/10 d 


Sorting Intermission 


Brought to you by last year’s students who claimed this lecture was too boring) 
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The Insertion Sort 


Well, we couldn't just teach 
you one sort, right? 


Let’s learn another! 


The insertion sort is probably 
the most common way... 


to sort playing cards! 


(But l'Il still explain the sort 
with library books) 
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The Insertio 


n Sort 


Let's focus on the first two mm mør 


books - ignore the rest. 
+ If the last book in this set 
IS 
in the wrong order 
e Remove it from the shelf 
+ Shift the book before it 
to the right 
+ Insert our book into the 
proper slot 


Great! Now our first two 
books are in sorted order 
(ignoring the others) 


The Insertion Sort 


= 
i 


f eee ee 


Ok, now focus on the first 
three books - ignore the 
rest. 
+ If the last book in this set 
IS 
in the wrong order 
e Remove it from the shelf 
+ Shift the books before it 
to the right, as 
necessary 


Gref NS RU UGH BER Ae 
book ape TF 'Sbrted order 


(ignoring the others) 
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The Insertion Sort 


= 
rT 


f= TE 


Ok, now focus on the first 
four books - ignore the rest. 
+ Ifthe last book in this set 
IS 
in the wrong order 
e Remove it from the shelf 
+ Shift the books before it 
to the right, as 
necessary 


° Insert our book into the 
Gr Pw eer first four 


books are in sorted order! 


| 


We just keep repeating this 
process until the entire shelf 
is sorted! 


The Insertion Sort 


So what’s the complete — 
algorithm? 
Start with set size s = 2 


While there are still books to 
sort: 


+ Focus on the first s books 


+ If the last book in this set is 
in the wrong order 


+ Remove it from the shelf 


+ Shift the books before it 
to the right, as necessary 


+ Insert our book into the 
proper slot 


"s=stl 


The Insertion Sort - Speed 


So what's the Big-O of our Insertion Sort? 


During each round of the algorithm 
we consider a larger set of books. 


During the first round, we may need to 
shift up to ome book to find the right spot. 


The Insertion Sort - Speed 


So what's the Big-O of our Insertion Sort? 
2 shift 


During each round of the algorithm 
we consider a larger set of books. 


During the first round, we may need to 
shift up to one book to find the right spot. 


During the second round, we may need to 
Shift up to two books to find the right spot. 


The Insertion Sort - Speed 


So what's the Big-O of our Insertion Sort? 


During each round of the algorithm 
we consider a larger set of books. 
N-1 shift 
During the first round, we may need to 
shift up to one book to find the right spot. 


1 step in round 1 


Dun ng Bie second rauno, we may NECULA i 5 ene Found 2 
+ 


shift up to two books to find the right spot. 


_ + N-1 steps in last rnd 
During the last round, we may need to = roughly N? steps 


shift up to N-I books to find the right spot. 


Thus, Insertion Sort 
IS O(N2), and is 
generally quite slow! 
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Insertion Sort - Bam Tu? 


right order too! 
Are there any kinds of Cy | ca | 
input data where ne 
Insertion Sort is either | 
more or less efficient? _ right orc 


Any ideas? 


Right! If all books are 
in the proper order... 


then Insertion Sort Conversely, a perfectly 


never needs to do any mis-ordered set of books 
shifting! is the worst case. 
In this case, it just takes roughly Si q 
7 I O(N) ince every roun 
N steps to sort the array! requires the 


maximum shifts! 


” The Insertion Sort EEEE 
Insertion Sort Questions 


And here's the C++ 
which sorts Can Insertion Sort be applied easily to 


sort items within a linked list? 


void insertionSort(int A[], int n iS RESEN SSR Sore a erable eon 


{ 
ae S = 2; S <= N; S+ i When might you use Insertion Sort? 
int sortMe = A[ s- 1 ]; 


| MAKE d COPY OI UNC IdSt Vad IM 
the current set - this opens up 
inti=s-2; a slot in the array for us to shift 


while (i >= 0 && sortMe < Alil} | it9®fift the values in the focus 


region right until we find the 
proper slot for sortMe. 
| | 


} Store the sortMe value into the 
HEE vacated slot. 

A[i+1] = sortMe; 
} 


-~ 


{ 
Ali+1] = A[i]; 


Sorting Intermission 


Brought to you by last year's students who claimed this lecture was too boring) 


But it's actually 
quite simple... 
And sometimes 
Simple is good! 


Everyone loves S/A% 
to make fun of le 
the: 


Ok, what’s the algorithm? 


Start at the top element of your array 


Compare the first two elements: A[0] and A[1] 
If they're out of order, then swap them 


Then advance one element in your array 
Compare these two elements: A[1] and A[2] 
If they're out of order, swap them 


Repeat this process until you hit the end of the array 


When you hit the end, if you made at least one swap, then 
repeat the whole process again! 


Bubble Sort In Action! Hic! 


Let’s see how bubble sort works with 
some of your favorite celebrities... 


SWAP? 


...to sort them based on the number of 
times they've been in... rehab! 


X 


If they're out of order, then swap them 


Compare the first two elements 


Compare the next two elements: A[1] and A[2] 
If they're out of order, swap them 


Compare the final two elements: A[2] and A[3] 
If they're out of order, swap them 


When you hit the end, if you made at least 
one swap, then repeat the whole process 
again! 


aim Bubble Sort In Action! Hic! 


Let’s see how bubble sort works with 
some of your favorite celebrities... 


SWAP? 


A 
4 sort them based on the number of 
times they've been in... rehab! 


Compare the first two elements 
If they're out of order, then swap them 


Compare the next two elements: A[1] and A[2] 
If they're out of order, swap them 


Compare the final two elements: A[2] and A[3] 
If they're out of order, swap them 


When you hit the end, if you made at least 
one swap, then repeat the whole process 
again! 


| gn Bubble Sort In Action! Hic! 


Let's see how bubble sort works with 
SWAP? some of your favorite celebrities… 


A 
4 sort them based on the number of 
times they've been in... rehab! 


Compare the first two elements 
If they're out of order, then swap them 


Compare the next two elements: A[1] and A[2] 
If they're out of order, swap them 


Compare the final two elements: A[2] and A[3] 
If they're out of order, swap them 


When you hit the end, if you made at least 
one swap, then repeat the whole process 
again! 


Bubble Sort In Action! Hic! 


Let’s see how bubble sort works with 
some of your favorite celebrities... 


SWAP? 


...to sort them based on the number of 
l times they've been in... rehab! 


Compare the first two elements 
If they're out of order, then swap them 


Compare the next two elements: A[1] and A[2] 
If they're out of order, swap them 


Compare the final two elements: A[2] and A[3] 
If they're out of order, swap them 


When you hit the end, if you made at least 
Wiren yooh ithe rene af \rauvinate arest 
one swap, then regeatrthe whole process 
And Wara done! 
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OK, so Bubble Sort 
has a bad wrap. 


er ey ERR on 
ae R EN 


~r nm Oy eo 
~~ ae 


Just like 
Insertion 
Sort, 
Bubble Sort 
is really 
efficient on 
pre-sorted 
arrays and 
linked lists! 


Bubble Sort Speed 


During each pass, we compare every 
element with its successor (and possibly 


Swap each). 
That requires about N steps. 


If we did even one swap, we need to 
repeat the whole process again. 


What’s the worst case? How many 
times might we have to repeat the 


process? 
Hint? Ok! 


Right! We might have to repeat 
this entire process N times. 


N passes of N “bubbles” = N2 


Ok, so Bubble Sort is O(N72)... 
But can it ever run faster in certain 


The Bubble Sort 


} 


void bubbleSort(int Arr[], int n) 


Bubble Sort 
Questions 
Can Bubble Sort be applied 


easily to sort items within a 
linked list? 


bool atLeastOneSwap; 


do 


atLeastOneSwap = false; 
for (int j = 0; j < (n-1); j++ 


Is Bubble Sort a 
“stable” sort? 


Is Bubble Sort ever faster 


if (Arr[j] > Arr[j + 1 
(Arr[j] [J ]) Ten 
Swap(Arr[J],Arr[j+1]); When might you use 
atLeastOneSwap = true; 
Bubble Sort? | 
} á 
} 
} If we swapped at least 
while (atLeastOneSwap == true) Le once, then start back at 
the top and repeat the 
whole process. 


Sorting Intermission 


ohmagif.cam 


Brought to you by last year’s students who claimed this lecture was too boring) 
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a » Challenge 
By one round, I 
mean one full trip 
through the sort’s je following 


J . 


211519 141713 


which has been sorted by 
one round of either selection 
sort, insertion sort or bubble 
sort. 


Which of these sorts could 
NOT have been used on this 
array? Why? 


selectionSort 


For each of the N books 
Find the smallest book 


between 


slots i and N 
Swap this smallest book with 
hook i 


insertionSort 


ESZ 

While books need sorting: 
Focus on the first s books 
If the last book in set is in the 
wrong order THEN 


A. Remove it from shelf 
B. Shift the books to 
the right as required 
C. Insert our book into the 
proper slot 
S=stl 


bubbleSort 


while the shelf isn’t sorted 
repeatedly swap adjacent books 
if they’re out of order 


Appendix 


° Shellsort - this won't be on your 
exam 


The sort eae 


Shellsort is based on an underlying procedt 
called h-sorting. Let’s learn h-sorting first... 


The method for h-sorting an array is simple: 


Pick a value of h 
For each element in the array: 
° If Ali] and A[i+h] are out of order then 
+ Swap the two elements 


lf you swapped any elements during the last pass, 
then repeat the entire process again (Same h value). 


The Shellsort: h-sorting 


Pick a value of h 


Let’s 3-sort this array so 


- If Ali] hesreellsheits C FE! 
e S are in order! = 


repeat the Atire proce) 
i i+3 


y v 


SWAP? 


The Shellsort: h-sorting 


Picka valueofh 
: | i Let's 3-sort this array so 


~~ shells are 


° If Ali] The’ The’ These shells ding. 
° S a al a al are in order! h=3 
| | 
If yous mA på aa E No swap! 
re swappctat D $ “70° © 
least once, 4 ” 
— repeat this i+3 


entire process! 


us bane we had/no swaps! _ = 
Our arrày is now 3-sorted! SO rti a g 
Pick This means that every 
Fer element 
is smaller than 


the element 3 items later 
Po in the array. 


at’s 3-sort this array so 
the shells are 
ascending. 
e.g., h=3 


Of course, it’s not Question: 


- completely sorted yet, just If you 1-sort an array 
-sorted! i i 
| pruecss! = so which other sort 


X algorithm does this 


O remind you of? 
SWAP? 


It’s required to 
always end with h=1! 
eg: 3, 3 . Ol 10,7, ~ 


ending with an h- value of 1: e. g. 8,4,2,1. 


Step 2: 
First completely 8-sort the array... 
Then completely 4-sort the array... 
Then completely 2-sort the array... 
Finally, completel bubble sort the array... 


and the array's now fully sorted! 


Each h-sort more correctly sorts the array, making 
the process simpler each iteration. 


Shell Sort Questions 


Can Shell Sort be applied easily to 
sort items within a linked list? 


Is Shell Sort a “stable” sort? 
What's the Big-O of Shell Sort? 


When might you use Shell Sort? 


The Shellsort 


Let's do an example on the board: 


Shellsort the following array using h values of: 3, 2, 
and 1. 


95 21437 


Sorting Challenge 


Given the following 
numbers, show what they 
would look like after one, 
two and three outer-loop 
iterations of selection sort, 
Insertion sort and bubble 

sort: 


9) (5) |2| 14 |3) 7 


selectionSort 
For each of the N books 
Find the smallest book 
between 
slots i and N 
Swap this smallest book with 
book i 


insertionSort 
caz 
While books need sorting: 
Focus on the first s books 
If the last book in set is in the 
wrong order THEN 


A. Remove it from shelf 
B. Shift the books to 
the right as required 
C. Insert our book into the 
proper slot 
S=stl 


bubbleSort 


while the shelf isn’t sorted 


repeatedly swap adjacent books 


if they’re out of order 


